dasLLAMA vulkan arc: decode-tail fusions, K-quant GEMV, memory-priority shield by borisbat · Pull Request #3581 · GaijinEntertainment/daScript

borisbat · 2026-07-27T18:01:54Z

The Vulkan tier arc: cooperative-matrix-2 production rails, the resident decode driver's structural levers, and the hardware capabilities the 5060 Ti exposes that we were not using. 26 commits, one arc, measured end to end on the same card against the same llama.cpp build.

Headline

row (Qwen3-4B, budget 6000, lcpp_bench 5-rep)	before	after	vs llama.cpp same card
Q8 tg128	77.85	80.42	89.8% → 92.8% of 86.66
Q4_K_M tg128	111.45	122.41	83.6% → 91.9% of 133.25
tinyllama Q8 tg128	261.78	277.27	89.8% → 95.1% of 291.43

Prefill is deliberately unchanged by the decode work (Qwen3-4B pp512 4379 → 4367, and 4358.10 vs 4357.87 on a same-session pair) — a useful negative control: only the decode record moved.

What landed

Decode-tail fusions. The decode tail is launch-bound — an add+rms is ~10us/layer for a 10KB row — so the lever is dispatch count, not kernel rate. Two new kernels: ar_add_rms_rq folds the Q8_0 requant into the add+rmsnorm that feeds it (the normed row quantizes straight out of ar_row, so xb never reaches device memory — it had no other decode consumer), and qkn_rope_kv folds per-head qk-rmsnorm into rope + KV store. A layer's activation quant now rides the PREVIOUS layer's addr_next, and the final norm needs no quant_final: 15 → 12 dispatches/layer.

Both reuse the split kernels' reduce order, amax fold and rounding verbatim and only skip a store/load round-trip, so they are bit-identical, not drift-class — the gate is exact hash equality rather than IDS parity. The implied per-dispatch drain (3.8us on Qwen's 108 removed dispatches, 4.8us on tinyllama's 44) lands inside the 3-5us band the phase was justified on, from two models with different layer counts.

K-quant decode GEMV → one lane per 32-block. The kq family still loaded one 4-byte word per lane (4 lanes/block = 128B in flight per warp), the shape q8_moe_groupn was ported away from earlier. A superblock's 32 quant words are 8 uint4s and index wsb*stride4 + blk IS block blk's four words (every per-format stride — 32/40/48 — divides by 4), so a lane owns a whole block: 512B/warp and one scale-plane decode per block instead of four. Drift-class, and slightly more accurate (one float fold per block instead of four).

memory_priority residency shield — the demotion cliff is gone. Measured on Qwen3.6-35B-A3B-UD-Q4_K_M, the model the cliff was characterised on:

budget	shield	pp512	tg128
13000	on	373.46 ± 0.83	35.48 ± 0.09
13000	off	370.39 ± 4.07	32.18 ± 2.36
14000	on	384.86 ± 1.04	35.70 ± 0.12
14000	off	384.85 ± 1.41	33.98 ± 2.19

The headline is the variance: ~20-25x less tg spread with the shield armed, which is the cliff's actual signature. pp is untouched (384.86 vs 384.85), so this is purely decode residency. And 14000 now beats 13000, where the pre-shield curve had 14000 collapsing — the budget can move past the old edge.

Dedicated transfer queue. Streamed expert copies ride a transfer-family queue signalling a timeline semaphore; the next compute submit attaches a GPU-side wait. Buffers are CONCURRENT-shared across the two families, and DASLLAMA_VK_XFERQ=0 keeps the single-queue rail. Qwen1.5-MoE pp +24%, GLM-4.5-Air pp +51%.

cm2 twin + arena (q8n native 34B blocks, decode-in-load prefill GEMM, q8n GEMV), the vulkan-flavor .dlim, the stream-slot reserve sized from the model rather than a constant, q51 tuner fixes, plane-stride truth consolidated into dasllama_gemm_schema, layout transforms extracted into dasllama_layout, and the router now defaults to the fastest GEMM path.

Negative results, kept deliberately

The k/v prefill roles get only 32 workgroups for 36 SMs, which looked like grid starvation. It is not: routing them to a wider grid loses — pp512 L128 4297 > M64 4101 (-4.6%, coopmat held constant) > sdot4-32 3906 (-9.0%). k/v is bound by the L-tile's arithmetic efficiency, so split-K is retired by implication, since its premise was the same "widen the grid, keep coopmat". DASLLAMA_MM_SMALLD ships as the instrument that settled it (default 64 = current routing, inert), with the verdict recorded at the knob so nobody re-runs it.

Also here

llvm -exe routes das main through a catch boundary, so a panic prints and exits nonzero instead of unwinding into nothing. The MCP stdio supervisor picks the newest daslang binary across build layouts (a stale one silently shadowed the current build and every native require failed with a build-id mismatch) and never lets child noise masquerade as a JSON-RPC frame — a compile error now surfaces as an error instead of dropping the whole server.

Gates

Bit-identity (PFHASH/DECHASH/IDS) fused vs split on Qwen3-4B and tinyllama, plus the banked tinyllama reference reproduced exactly; paranoid==elided on the fused rail; spirv-val VALID on all new kernels; test_vulkan_tier 34/34 (its kq/q40/mixed arms are what cover the ported GEMVs against CPU references); test_vulkan_tier_cm2 6/6; lint and format clean. All new env knobs are registered in dasllama_env and read through the typed readers, resolved once rather than per step.

New shared module owning the metal-blob transform, the CPU repack walkers (RepackReg + q8/kq/mx4/q51), and the GPU tier gathers (q8 + kq) — moved verbatim from dasllama_common. The loader reaches the six entry points it calls through [init]-registered hooks (register_model_layout, same shape as register_metal_servable); unset hooks panic, and the transformer facade + dasllama_image both require the module so every load path registers. Consolidations riding along: metal_blob_fmt_at (byte-identical fmt_at dup) deleted; fmt_at + dn_layer_q8 made public; LINT014 unmasked two stale `var` Model params (resident_upload, moe_gpu_upload_resident — never written, the old gathers' unsafe addr chains had hidden it). Gates: test_vulkan_tier 34/34 with [I] tier lines (real vulkan engagement), lint 0 issues across the 9 changed files, das-fmt applied. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

`daspkg release --root modules/dasLLAMA/benchmarks` now produces a self-contained dasllama-bench/ bundle (exe + dasModuleVulkan.shared_module + runtime DLLs + per-box tune sidecar) — the ladderboard's deployable unit for this and future boxes. Acceptance on the 16GB 5060 Ti: 20.9s wall for a full pp512/tg128 5-rep row with all vulkan tier lines present; pp 4306.32/tg 77.94 matches the -jit recipe within noise. The per-row compiler tax (~103s) and the q51-bug tune-relaunch (~6min) are gone from the measurement loop. Note: dasllama-server's release_requires_jit() rationale ("a baked -exe mis-wires the vulkan GPU-tier arm probe") did not reproduce here — the baked exe wires the tier correctly. Likely stale since the 2026-07-12 exe fixes; server deploy revisit tracked separately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

@13000

…t a constant The 1.1GB STREAM_RESERVE carve was a fixed guess; actual slot need is the largest streamed expert layer's device planes (2x gate/up + down), which is model geometry. Qwen1.5-MoE needed 1103MB — failed the assert by 3MB, so the full model never ran; GLM-4.5-Air needs 2x1.56GB and could never fit the constant. The 35B cleared it by 148MB of luck. Loader now reports the exact need before the first upload (new loader-contract set_moe_gpu_stream_need in dasllama_math; moe_gpu_upload_resident computes the max over expert layers via the existing moe_gpu_plane_bytes hook — no stride duplication), and the tier carves exactly 2x need at init, falling back to the legacy constant only for direct-upload paths that never ran upload_resident. The ensure_stream_slots assert now checks the carved reserve and stays as a true invariant. Validation (16GB 5060 Ti, mm, 5-rep): GLM-4.5-Air Q4_K_M @13000 runs first time ever on this rail — pp512 55.78 / tg128 6.74, 42/45 layers streamed, slots 2x1.56GB; Qwen1.5-MoE Q8 @6000 runs first time ever — pp512 320.13 / tg128 35.03, slots 2x526MB. Lint clean; test_vulkan_tier 33/34 — the one red (attention npos=1100) is pre-existing on the box today, proven by stash-bisect, tracked separately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

…eting a locked array Two bugs in gen_tune_probe's mismatch-abort rail (found via the q51 mr4 MISMATCH on zen2, maxdiff 120.2): 1. The abort deleted `tvs` from inside `for (v in tvs)` — the locked-array trap — so every gate failure died with EXCEPTION rc=1 instead of aborting cleanly. The abort now flags + breaks and the shared post-loop cleanup runs. 2. A gate-failed family left its manifest key MISSING, so scope completeness failed and every subsequent run re-tuned from scratch (~6 min per bench row, per daspkg release, forever). Gate failures now record "reference" (the always-correct original body — an explicit reference entry counts for completeness per llvm_tune) with a loud "gate-failure fallback" line, for both the kq family loop and the q51 family. Deliberately NOT the fallback perm: for q51 the fallback chain names mr4, the very perm that mismatches. Validated: gen_tune_probe rc 0, q51 sweep 104ms, manifest gains "q51q8_gemv_gen":"reference", no exception. The mr4 kernel-gen mismatch itself remains open (fix-list #3) — now quarantined behind a correct stamp instead of a crash loop. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

…anion registry The "q51 mr4 MISMATCH (maxdiff 120.2)" on zen2 was the PROBE's bug, not the kernel's: on x64 the mr4/mr8 perms correctly decline (no sdot) to the reference body, which reads its interleave from the layout companion (grp1 in the tuner process) — but q51_mr_of trusted the SUFFIX, repacked the planes to grp4, and graded a grp1-walking kernel against grp4 planes. The kq families already resolve mr per-variant off the companion registry ("declined perms report the reference resolve — the lockstep rail"); q51's helper predated that rail. Now mirrored: q51_layout_mrs() off q51q8_layout_gen_variants(). Validated on zen2: reference/mr4/mr8 all pass at maxdiff 3.8e-06 (all reference-tier here), family 158 ms, real winner recorded — closing the 6-min re-tune loop's root cause. Production was never affected: x64 backend wiring routes q51 to the portable disk-order kernels regardless. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

Commit 2 of the layout-extraction track: the schema (the declared "one stride source") gains the q8 device-plane row (Q8_BLOCK_ELEMS/Q8_QPB/Q8_SPB), the kq device constants (KQ_SUPERBLOCK_ELEMS/KQ_DEV_SSB), and the q5_1 strides (Q51_QPB/Q51_SPB). Consumers reroute: - dasllama_layout: the kq gather's inline stride ternaries (160/192/128, 18/16/20) become kq_qsb/kq_ssb via a new kq_schema_id(KqFmt) — the ONE KqFmt->schema-id bridge; the q8 gather, metal-blob transform (34B = QPB+SPB), and repack_regions' q8/q51 offsets reroute to the named constants. - dasllama_math_vulkan: requires the schema; arena_block_bytes drops its int(KqFmt)-keyed stride copy for kq_qsb via a named vk_kq_schema_id bridge (the tier's int fmt space cannot see KqFmt — retyping VkStack.fmt is noted remaining debt); arena_blocks_for uses the shared block-elem constants. - dasllama_common: rdec_blocks_for joins. Metal's restatements stay untouched (PR #3570 owns those files). Gates: full chain compiles; lint 0 across 4 files; test_vulkan_tier 34/34 (the npos=1100 red cleared spontaneously — transient post-pressure driver state, tracked); tinyllama parity BIT-IDENTICAL to the pre-change banked pair (PFHASH 0x51ffa7983d54 / DECHASH 0x56331e1ccf49, interp apples-to-apples). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

Boris directive ("once settled — the fastest path should be the default"): coopmat_mode now defaults to mm (the L-tile coopmat twin) on coopmat-capable devices instead of requiring DASLLAMA_COOPMAT=mm — phase-0 measured the difference at 4297 vs 1541 pp512 on Qwen3-4B, and an unarmed bench silently measuring the fallback is exactly the recipe-miss class this kills. The env stays as the override/bisect hatch, with a new "sdot4" value pinning the int-dot fallback explicitly. test_vulkan_tier pins mode 0 via a new vk_force_coopmat_mode test hook ([init], before the lazy device init): its CPU-reference arms' tolerances were derived against the sdot4 reference kernels, so the suite stays deterministic under any router default (the earlier flip attempt turned the dn arm red at npos=1100 for exactly this reason — fp-accumulation-scale drift, not a kernel bug). The coopmat kernels' numeric envelope remains covered by the drift-class model gates. Un-stashes the parked flip; both its blockers cleared (the npos=1100 box-state red recovered spontaneously, and the suite no longer depends on the router). Gates: lint 0, tier suite 34/34 with the flip live, bench smoke engages mul_mm with no env set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

One process benches every listed model in turn, so a sweep pays the compiler/ startup cost once instead of per row (~103s/row under -jit before the exe rail; still one JIT+load per sweep after). Per-model: load on the image rail, bench rows, free, then the optional llama-bench ref — one model in memory at a time, per the protocol. md output aggregates all models into one table (model column already per-row); -o json emits one BenchModel per gguf in the existing array shape. A single -m path behaves exactly as before. Smoke: tinyllama + Qwen3-0.6B in one process, both rows produced, exit 0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

… exit nonzero A standalone exe's generated entry called das main with NO catch boundary, so a runtime exception unwound into nothing: no message, an opaque exit status (observed as the STREAM_RESERVE assert exiting 127 from the released bench exe while -jit printed a proper EXCEPTION). Fleet-fatal — a rented box must scream, not die silently. New DAS_API jit_run_main_guarded(ctx, mainFn, resultKind) in module_jit.cpp — beside jit_run_web_lifecycle, the established fn-pointer-helper pattern — wraps the call in Context::runWithCatch (correct under both the EH and longjmp runtime flavors), prints "EXCEPTION: <message>" on failure, and returns 1; resultKind maps void/int/bool mains (bool called through a bool(*) cast — AL's upper bits are undefined, an int32 read would garbage the branch). The generated entry (llvm_exe.das) routes through it and keeps jit_shutdown before the return. Acceptance: panic fixture -exe prints "EXCEPTION: boom ..." and exits 1; healthy int-main fixture prints and exits 42. Build note: the Dyn targets hit the known /Z7 LNK1103 on the first incremental — clean-first on both, per the playbook. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

…, and lens rails Everything the cm2 production twin needs underneath it, landed green before the kernel itself: - KqFmt.q8n (= 6): the same q8 weights as gguf-native 34B interleaved blocks ({f16 d; int8 qs[32]}, ONE plane) — a GPU-tier tag, never a disk format. Schema gains Q8N_BPB; the tier's arena/stack math grows the q8n arm (single plane, ws = 0, 32-elem blocks via vk_fmt_b32). - Device + mode: has_coopmat2 probe, DASLLAMA_COOPMAT=cm2 -> mode 4 (guarded on the extension, downgrades to mm without it), the cm2 device chain via #77's creator. cm2 stays OPT-IN until it beats mm in the bench — the fastest-path default is measurement-driven. - dasllama_layout: moe_gpu_gather_stack_q8n — prepared planes (grp<mr> or row-major, wbias XOR) -> interleaved 34B blocks; hook chain extended (GatherStackQ8nFn, register_model_layout arity +1). - Kernel-access lens: the cm2 tensor intrinsics modeled (coopmatStoreTensor writes arg 1; coopmatLoadTensor/Decode plain reads) on top of the merged #3570 analyzer. Gates: full chain compiles, lint 0 across 5 files, test_vulkan_tier 34/34. Branch rebased onto master (post-#3570/#3571) with a full C++ rebuild and a fresh dasVulkan shared_module. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

@6000

…e stack path The production half of P5b: mode 4 (DASLLAMA_COOPMAT=cm2) now serves fmt-6 q8n expert stacks end to end. - q8n_moe_batch_cm2: the P5a ladder winner (n2x256 - two 64-wide token tiles per 256-invocation workgroup, ONE A decode per 128 output columns, token tiles fastest) grafted onto the batch REGION/META skeleton. A = 34B blocks decoded in-load by the [spirv_decode] function; B = f16 activations; stores ride the transpose view into the token-major y. Layout dimensions bound every access to the region - and the output layout MUST be the clamp-Constant type: only clamped layouts DISCARD out-of-bounds stores, an Undefined-clamp store from a partial token tile stomps the neighboring region's rows (lcpp's own store layout is clamp 1; the aligned-shape probes could never catch this). - q8n_moe_groupn: P4's native-34B GEMV arm (17 u16/block, pair loads, d broadcast-read) on the groupn region skeleton. Decode stays on the int-dot path - Q8_0 activations, the requant mid-chain, int-exact vs reference. - The f16 feed: q8_moe_xf16 dequantizes the uploaded Q8_0 image into the f16 plane the twins load as B (ffn-meta word 3 carries the word count); q8_moe_actf16 replaces actrq on the cm2 route (act(g)*u straight to f16, no quantization). Chain branches key on the PER-STACK format, so mixed triples compose (q8 gate/up + q8n down and the reverse). - Router: ONE helper (moe_gpu_expert_fmt) maps q8 expert stacks to q8n when the tier asks (moe_gpu_expert_q8n probe hook), applied at upload, dispatch, and heat-advise so they can never disagree. Mode 4 keeps the whole mm ladder for fmt-0 stacks. gather_upload grows the q8n arm (EMPTY ws); upload_stack takes a null scale plane; make_stack_shell floors the ws buffer at 64B for its descriptor slot; stream mirrors and heat slices skip the absent ws copies. - fill_batch_meta tiles the twin's two axes (64 weight rows x 128 token columns); fmt_unit/vk_fmt_b32 replace the fmt!=0 unit checks on the paths a q8n stack reaches. Gates: new test_vulkan_tier_cm2 (own process - pins mode 4 where the main suite pins sdot4) 4/4: decode int-exact at 1e-3, batch/multi-tile/mixed/combined/ streamed at the f16-accumulate 2e-2 bar; spirv-val VALID x4; tier suite 33/34 both ways (the 1 red = the known npos=1100 transient, identical with the diff stashed); Qwen1.5-MoE Q8 @6000: cm2 IDS == mm IDS token-identical x32 (vs pure CPU one flip at a genuine 0.05-gap near-tie that mm makes identically; the ~2ulp act-vec control keeps the CPU token - the twin sits inside the shipped default's accepted envelope); paranoid == elided BIT-IDENTICAL; bench pp 322.56+-2.29 vs mm 320.13 (wash), tg 33.85 vs 35.03 (the P4-measured native-34B GEMV delta). cm2 stays OPT-IN until it beats mm - the win is expected from the arena slice (qwen3-4B rides the resident driver, not expert stacks). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

@6000

…e f16-fed twin prefill Dense q8 models under mode 4 (DASLLAMA_COOPMAT=cm2) now place their planes as q8n (fmt 6) in the resident arena and run the whole prefill on the cm2 twin: - loader: resident_place grows the q8n gather arm (single interleaved plane, EMPTY ws); resident_layer_fmts promotes q8 -> q8n per activation-group ({q,k,v} and {gate,up} whole-group, wo/down/cls singletons) so members sharing one B image always agree on its form - arena: ws buffers floor to 64B for the scale-less fmt (stack-shell rule), vk_arena_place accepts the empty-ws q8n contract, fill_arena_batch_meta learns the twin tile rule (64 weight-rows x 128 tokens) - prefill chain: new rd_f16cvt kernel (f32 rows -> f16 plane) replaces the quant roles on promoted groups, actf16 replaces actrq ahead of a q8n down, GEMM roles route batch_cm2 with matching meta tiles; decode GEMVs and cls ride the existing q8n groupn (Q8_0 activations, unchanged) - seam fixes: vk_arena_gemv/vk_arena_ffn/vk_moe_dense derived their activation unit from fmt != 0 - q8n keeps 32-block Q8_0 acts, now via fmt_unit/vk_fmt_b32 Gates: cm2 test 6/6 with the new arena arm (q8n place + arena GEMV int-exact); tier suite 33/34 (the 1 red = the known npos=1100 box-state transient, same signature with the diff absent); mm path proven untouched (tinyllama parity reproduces the banked PFHASH/DECHASH bit-exactly); cm2 IDS == mm IDS token- identical x32 on tinyllama AND qwen3-4B (qk_norm chain); paranoid == elided bit-identical. Bench (qwen3-4B Q8 @6000, 5-rep): cm2 pp 3110.72 +/- 43.96 vs mm 4269.27 +/- 211.32, tg 72.07 vs 77.76 - the single n2x256 tile loses on the d=2560 roles (wo +68%, down +53% by GPU_PROF) exactly where P5a predicted the ladder is needed, so mm STAYS the default and cm2 stays opt-in until the tile router lands. Side-win banked: q8n upload is a straight memcpy - 10.3 GB/s vs 3.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

Every device-local allocation now chains VkMemoryPriorityAllocateInfoEXT at priority 1.0 (default is 0.5) on both alloc paths (ReBAR raw + boost), so WDDM pressure demotes other apps allocations before the weights/KV/scratch. Gated on the coopmat creators (both enable VK_EXT_memory_priority opportunistically per the dasVulkan side) + memory_priority_supported; DASLLAMA_VK_MEMPRIO=0 is the bisect hatch. This is the correctness shield for the recurring post-pressure npos=1100 transient (fix-list #5) - the demotion-cliff class phase 0 measured on the 35B budget curve. Inertness proven: tinyllama parity reproduces the banked mm hashes bit- exactly with the shield armed (PFHASH 0x51ffa7983d54 / DECHASH 0x56331e1ccf49); cm2 test 6/6. The pressure A/B (GLM Air partial-fit + desktop load) waits for a quiet-box session. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

@6000

… (phase 5) Streamed mirror->slot fills now submit on the device transfer family, signaling a timeline semaphore; the slot consumer hands the value to the next compute submit as a GPU-side wait (one wait covers the whole in-order chain). Copies genuinely overlap compute instead of serializing on the one queue. Stream mirrors and slot buffers go CONCURRENT-shared across the two families (heat slices still read mirrors on the compute queue); the cold-miss sync path rides the transfer queue too and host-waits the timeline. Model drop drains the transfer timeline before the buffer sweeps. Single-queue mode (no dedicated family, no timelineSemaphore, or DASLLAMA_VK_XFERQ=0) keeps the old rail verbatim. Gates: cm2 test 6/6 + tier 33/34 (the 1 = the known npos=1100 box transient) with the queue armed; Qwen1.5-MoE parity IDS == banked x32 and hashes BIT-IDENTICAL between armed and single-queue runs; drop test passes both ways. Same-build A/B (5-rep / 3-rep): Qwen1.5-MoE Q8 @6000 pp 403.14 +/- 2.30 vs 325.23 +/- 1.17 (+24%), tg 33.87 vs 34.70 (the tail prefetch now overlaps the first decode tokens - net hugely positive); GLM-4.5-Air Q4KM @13000 pp 75.98 +/- 0.49 vs 50.18 +/- 2.28 (+51%), tg 6.43 vs 6.49 (wash). The 1.5GB/window DMA-overlap poster child delivers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

@13000

…gs name the reason The heat gate required the DOWN stack to match the gate/up activation form - but the dispatch rails have served mixed triples (k4 gate/up + q8 down, the GLM Air shape) since the beginning, with the mid-step form following the down stack. Relax to the real constraint (gate/up share one act image; down free). The skip log now names WHICH gate refused (want 0 / no mirror src / gate-up mismatch) instead of a blanket 'no stream mirrors'. Found chasing GLM Air tg: every one of its 42 streamed layers was silently heat-ineligible. With the fix, DASLLAMA_GPU_HEAT=8 @13000 engages 8x8+1 slots (budget-clamped) - measured tg-neutral at that coverage (6.02 +/- 0.43 vs 6.43 baseline; 830MB pinned vs a 6.8GB/token read stream), so the fix is a capability unlock, not yet a win. Tier 33/34 (the known transient), heat + mixed + streamed arms green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

…e them back at load .dlim is a local per-config bake (Boris's ruling - never portable). The vulkan flavor stores what the vulkan serving plan needs: the planar planes the CPU side reads PLUS every gather output of the GPU walk, in walk order: - Model grows vkblob (concatenated device-layout planes, 4KB-aligned starts for the coming import-window rail) + vkplan (VkPlanEntry walk-order table); both ride the generic image plane walk - NO format change, NO version bump (old planar images lack the sections and stay valid) - the three gather wrappers grow collect/slice arms: a normal load's GPU walk COLLECTS outputs into the blob (bake is free - the upload proceeds as usual), a flavor load's walk SLICES the mapped blob back instead of transforming; any plan divergence declines to live gathers mid-walk (seamless - matched entries were byte-identical) with a stale-image warning, since the walk shape depends on more than the tag folds (seq_cap moves the resident split) - identity: new moe_gpu_bake_tag hook (vulkan registers env-or-auto knobs ONLY - resolving the device at identity time carved the stream reserve before the loader reported the model's slot need, resurrecting the pre-518ae164a 3MB assert); tag folds mode/vram/layers/stream/rail pins, any config switch misses the identity and rebakes - load_model_cached: want_vulkan branches like want_metal (gguf path bakes AFTER the planar save so planar stays planar; planar-image loads bake too; direct-.dlim tries the vulkan identity first when the tier arms) - collector: 64-bit throughout (the blob crosses the int32 array rail at Qwen1.5's 2.15GB; the slicer indexed the blob through int() - truncation), growth capped at 2GB steps past 4GB (doubling's overshoot paged a 128GB box on GLM's bake) Gates (all hashes bit-identical between gather-loaded and sliced runs): tinyllama bake 2.4s, slice load 1.09s (planar ~5s); Qwen1.5-MoE bake 31.3s, slice load 6.7s (the gather path spends ~30-80s); GLM-4.5-Air 98GB image, cold slice 92.9s / warm 38.2s (gather baseline 25.7s ran warm; GLM's real load win arrives with the P2 imported-mirror + spill rail which removes both 21GB unevictable copies). Lint 0/4, das-fmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

…ve+parallel The GLM bake forensics (45 min, 65 MB/s effective): NOT disk, NOT memory (256GB box) - the gguf path O(model) das loops run interpreted in a no-jit process. Measured on tinyllama: load stages 53s interp vs 5.5s jitted, with repack itself 0 ms EITHER way - repack_regions is already jobque-parallel and invokes llvm_tune native kernels that compile regardless of -jit, so the interp cost lives in the weights/meta decode loops and the GPU gathers. guard_interp_gguf_load: a gguf over 4GB refuses to load without jit_enabled() || is_standalone_exe() (the server rail idiom), naming the fixes - run -jit, load a prepared .dlim (the slice path stays interp-friendly, 6.7s Qwen1.5), or DASLLAMA_ALLOW_INTERP_LOAD=1 for a deliberate slow load. Also corrects the bake-growth comment (the overshoot wastes RAM; it never paged the 256GB box). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

…ar inscope Load-lifecycle audit (Boris suspected undeleted arrays after repack). The repack chain itself is CLEAN: regs deleted in all four walks, qscales freed after the f16 convert, gather wq/ws deleted in both upload walks, the gguf mapping closes with its fmap block, the metal-blob convert frees all three original planes, and the shard-merge deletes per-shard metas. What leaked: the parsed GGUFMeta (kv-location table + tensor directory + name strings, ~1-2MB per parse) at NINE sites - the single-shard model load (the shard path deleted, the common path did not), both tokenizer parses (SPM load + kind sniff), the BPE loader, and the four audio-tower/mmproj sniffers - plus the SPM/BPE vocab temp arrays (toks/scores/merges, several MB on 150k vocabs). Per-load MBs, accumulating across model swaps in a server process. All nine now declare var inscope (finalize-at-scope-exit, per Boris - the syntax serves plain structs/arrays now). Gate: interp gguf-path load reproduces the banked parity hashes bit-exactly (0x51ffa7983d54/0x56331e1ccf49); lint 0/7, fmt clean. (A -jit gguf load produces DIFFERENT hashes by design - the probe compiles with _jit_fast_math; the parity protocol stays interp.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

Two lines claimed inscope was smart_ptr-only; Boris corrected it and the gguf-meta audit compiled + ran clean with var inscope on plain structs and string arrays. The AST-pointer prohibition stands unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G1Qe2TFnnxaLv4TyQm9Xeh

…rsion profiling The GLM-4.5-Air draft head was disabled at load because the file ships nextn.embed_tokens/shared_head_head and we assumed those were head-private weights. They are not: embed_tokens is BYTE-IDENTICAL to token_embd, and shared_head_head is output.weight at a coarser quant (q4k vs the trunk's q6k, cos 0.9973 over 2048 rows sampled across the whole vocab; the control, output vs token_embd on this untied model, reads cos -0.025). DeepSeek-V3-style MTP shares both with the main model by construction, which is what "shared_head" names. So the head drafts through the TRUNK's tables (same weights, better classifier) and the copies never load; a shape mismatch still fails closed. Two bugs this surfaced: - qkv/out bias arrays were sized and filled with the TRUNK layer count while every other per-layer walk uses layers_ld. glm4moe is the first arch combining attn_qkv_bias with a NextN block, so blk.<n_layers>.attn_{q,k,v}.bias never loaded and the draft read past the end of t.bq. - prefill_attention gated head-parallelism on a work threshold while attention_std_decode threads over heads unconditionally, so a 2-row verify (strictly MORE work than the npos=1 decode that threads for free) fell under the gate and ran single-threaded on 16 lanes. Profiled: attn 38% -> 6% of the window, dispatches 109 -> 908, inline 799 -> 0. GLM MTP 0.62x -> 0.98x, Qwen 0.8B 0.77x -> 0.89x, acceptance byte-identical on both models (pure threading change). IMAGE_VERSION 5 -> 6. The image identity is computed BEFORE the gguf is parsed, so it can carry box/knob state but nothing config-derived; a loader change that alters which tensors load leaves a structurally VALID stale image that silently represents a different model. Only the version catches that, and the contract comment now says so. A rejected draft no longer re-forwards on non-recurrent archs: the verify already produced row 0's logits (softcap+suppress mirrored by construction) and its KV row is correct, so the step consumes them and re-seeds the draft head from mtp_h0, the row-0 normed hidden both classifier paths already compute. Recurrent archs still restore and re-advance, since there the verify genuinely advanced state past the draft. Conversion profiling behind DASLLAMA_CONV_PROF: load_big is the choke point every big tensor rides and its branches ARE the conversion kinds, so bucketing there answers where gguf->image time goes. On GLM (73 GB, cold) it accounts 89492 of 89946 ms and shows transcode is ~5% of the total; cold conversion is page-fault bound on the mapped source, warm is bound by the image write. test_mtp gains a glm4moe arm: the only non-recurrent MTP model we have, so the sole coverage of the reject branch above, and the arch where the bias bug lived. It uses a CODE fixture deliberately. Strict token-identity is only meaningful on a near-tie-free prompt, since the accept path reads batch-kernel logits (mm_gemm/mm_b_f) while plain decode reads GEMV (mm_kq) and those differ in reduction order, which with_pinned_kernels does not cover. At Q4_K that delta sits below the model's own quantization error, so tie-dense prose diverges with no defect present. Follow-up noted in the test: a margin-gated arm can cover prose. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETtsmrLWEZh83f99SY3fqX

…mtp-ab, --prof The bench card claimed cmd was "the reproduction receipt: the exact invocation" but lcpp_bench RECONSTRUCTED it, hardcoding "bin/daslang -jit .../lcpp_bench.das" plus re-serialized args. That names the wrong binary and, more importantly, captures NO environment. Half the A/B levers are env-armed by design (--qkv-ab needs DASLLAMA_GPU_QKV=1, --dense-ab needs DASLLAMA_GPU_DENSE=1) and every run carries DASLLAMA_BOX / DAS_JOBQUE_THREADS / DAS_TUNE_MANIFEST, so a card holding --qkv-ab without its arming var is not reproducible, and two configs differing only by an unrecorded env var look like the same row. bench_cmd_line() now returns the real process argv and bench_env_line() captures every engine/tune var that is set (the whole set, not a curated subset: an unrecorded var is exactly the failure mode). BenchRun gains `env`. Both live in profile_common so any bench inherits them. A/B levers were four parallel hand-built chains: a mutual-exclusion count, the `ab` flag, per-lever env warnings, and a dtag/setter ladder. They collapse to one Lever struct plus levers_of() - flag, tag, routing fn, required env. Adding a lever is now one row, which is what makes "pull an archived bench in when we need it" cheap. The unarmed-lever warning is derived rather than restated per lever; that case matters because an unarmed lever measures the same thing twice and reads as a real 1.00x. --mtp-ab is the first lever pulled through it, from the archived decode_real_bench. It needed one new concept: Lever.real, meaning the lever is only meaningful under REAL greedy continuation, so it runs a tg-real{n} row instead of the llama-bench tg row. Synthetic ids would make acceptance meaningless while still producing a plausible number. Fails closed when the model has no NextN head, since set_mtp_spec silently no-ops there and both arms would be identical. --prof runs an untimed forward_profile window per arm; with --mtp-ab the off/on bucket delta is what attributes draft vs verify vs classifier. That is how the single-threaded verify attention was found. write_cliff_probe.das: sustained-write throughput as a time series, no model, no jit, no tune. Written to check whether the .dlim writer's 275 MB/s was the drive. It is not - the same volume sustains median 2044 MB/s over 50 GB with three isolated writeback stalls and no degradation, so the image writer is ~6x off the hardware for reasons still unknown. Kept as an instrument rather than a scratch probe because the answer it gives is not reproducible from a conversion total. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETtsmrLWEZh83f99SY3fqX

…SON-RPC frame Two bugs that together made the daslang MCP server vanish from a session with no diagnostic anywhere the client could see. 1. daslang-mcp-msvc.cmd picked the binary first-wins, preferring the single-config bin\daslang.exe over bin\Release\daslang.exe. A box that has built both layouts then runs whichever is older: here a Jul-18 0.6.3 build (DAS_BUILD_ID 60308) shadowed the current 0.6.4 one (60408), so every dynamic module reported a build-id mismatch and utils/mcp/main.das died on `error[20605]: missing prerequisite 'dashv'`. Fix: _pick_binary() takes the NEWEST of bin/Release, bin/, build/, build/bin — one implementation, re-resolved per spawn so a mid-session rebuild can switch layouts, handed to the .cmd via DASLANG_MCP_BIN. Setting DASLANG_MCP_BIN externally pins an explicit binary as a bisect hatch. The .cmd keeps a standalone fallback, now Release-first. _daslang_binary() (the dasherd .mcp.json entry) shares the same rule. 2. DaslangChild._read_line() returned the child's first non-empty stdout line as the response. A daslang compile failure prints there, so the error became a corrupt frame and the client dropped the entire server rather than surfacing it. Non-JSON lines are now logged, skipped, and carried in the ChildDead message, so the failure arrives as a JSON-RPC error containing the actual compiler output. Verified both paths: healthy spawn serves the full tools/list; pinned to the stale binary the tool call now returns `supervisor error: eof; daslang child said: error[20605]: missing prerequisite 'dashv' ...` instead of silently disconnecting. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

…ne pass each Phase 6 of the vulkan arc. The decode tail is launch-bound (an add+rms is ~10us/layer for a 10KB row), so the lever is dispatch count, not kernel rate. Two new kernels, both decode-only: * ar_add_rms_rq — ar_add_rms + dn_requant. The normed row quantizes straight out of ar_row into xq/xs, so xb never reaches device memory; it had no other consumer in the decode chain. eps/ascale move to a 2-float tail on the norms buffer because binding 4 is the scale output now, and binding 5 gains a uint view (vk_yq) for the quant words. 8 lanes per 32-block, 32 blocks per pass — the 8 lanes of a block are consecutive and 8-aligned, so the xor-1/2/4 amax fold only ever exchanges with co-active lanes even when other groups have left the loop. * qkn_rope_kv — qk_rms + rope_kv_store (qk_norm models only). One wg per head-row, qk_rms's own dispatch shape: the head normalizes into qr_row and ropes from there. The per-layer rms_q/rms_k rows and eps ride the cos/sin buffer, since binding 3 is the V mirror here — the same packing prefill's at_prep_core already reads off binding 4. Both reuse the split kernels' reduce order, amax fold and rounding verbatim, and only skip a store/load round-trip, so they are BIT-IDENTICAL rather than drift-class. Effect: a layer's activation quant rides the PREVIOUS layer's addr_next and the final norm needs no quant_final, so 15 -> 12 dispatches/layer on qk_norm models, 14 -> 12 otherwise. DASLLAMA_VK_FUSE=0 pins the split rail for a same-build A/B; the profiler's role table follows the active rail (arrq_f/arrq_n/qknrope vs rq_x/rq_f/qkn). Also fixes a latent silent-corruption bug in vk_rdec_prepare: it never checked dim against AR_MAX_DIM or head_size against 256, so a 13B-class dim of 5120 would write past ar_row's 4096-float workgroup slab with no diagnostic. Now declines fail-closed, same shape as the existing KV-mirror guard. GATES: Qwen3-4B Q8 fused vs split BIT-IDENTICAL (PFHASH 0x1b1c82c24034d / DECHASH 0x1979cb8ec6120, IDS token-identical x32 — both fusions live); tinyllama Q8 reproduces the banked 0x51ffa7983d54 / 0x56331e1ccf49; paranoid==elided bit-identical on the fused rail; spirv-val VALID x2; test_vulkan_tier 33/34 (the 1 red is fix-list #5's npos=1100 transient, maxdiff 4.706375 — the exact banked signature, proven diff-absent twice before); test_vulkan_tier_cm2 6/6; lint + format clean. Perf deliberately NOT claimed here: the parity probe's run-to-run spread is +-9%, which swamps the effect. The judged tg number rides lcpp_bench's 5-rep protocol. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

@6000

Phase 7's tg half. The q8 GEMV took this lever back in the 4-lever session (uint4 16-block span, +4-12%); the K-quant family never did, and it was still loading one 4-byte word per lane — 4 lanes per 32-block, so only 128B in flight per warp. A superblock's 32 quant words are 8 uint4s, and uint4 index `wsb*stride4 + blk` IS block blk's four words (every per-format word stride — 32 q4_0/Q4_K, 40 Q5_K, 48 Q6_K — is divisible by 4). So one lane can own a whole block, which buys two things at once: * 4x the bytes in flight per warp (512B vs 128B), the actual lever * one scale-plane decode per block instead of four — the old shape had all four lanes load and unpack the same dm/sc/mn Nibble pairing is unchanged: weight word j's LOW nibbles dot x word j and its HIGH nibbles dot x word j+4, and a block's 8 activation words are exactly two uint4s (lo half, hi half). Added a uint4 view of the activation binding (vk_xq4) beside the existing vk_wq4. Shared helpers: kq_blk_sum (the activation sum the min/offset terms fold against), kq_blk_dot (the 4-bit dot), k5_blk_dot (same fold plus the 5th bit deposited from the block's qh word), k6_blk_dot (halves kept apart — the lo/hi nibble halves ARE Q6_K's two 16-weight sub-scale groups, so each needs its own dot and sum; returns an int4 because a shader function cannot return a tuple). Per block the integer dot and activation sum are now exact 4-lane merges, but the float term folds once instead of four times, so this is DRIFT-CLASS, not bit-identical — and slightly more accurate (fewer roundings). MEASURED (lcpp_bench 5-rep, Qwen3-4B-Q4_K_M @6000, box quiet): tg128 116.55 -> 122.41 = +5.0% (before 116.63+-0.08 / 116.48+-0.09 / 116.54+-0.02; after 122.50+-0.07 / 122.32+-0.15) pp512 wash (4574.00 -> 4573.53) — correct, the batch GEMM is untouched With phase 6 this row has gone 111.45 -> 116.55 -> 122.41 t/s = 83.6% -> 91.9% of lcpp's 133.25 on the same card. GATES: test_vulkan_tier 34/34 — the kq (k4 gate/up + k5/k6 down), q40 (all-Q4_0 triple) and mixed (k4 + q8) arms each dispatch these kernels and compare against a CPU reference, which is the real correctness gate here; spirv-val VALID on all four; paranoid==elided bit-identical; lint + format clean. The npos=1100 transient did not recur this run. NOTE for whoever next reaches for it: _vk_prefill_parity.das is a DEGENERATE fixture on Q4_K_M — it reports PFHASH == DECHASH exactly (0x128bffffdae80), i.e. the logits do not change across all 32 decode steps, and IDS is all zeros. Verified pre-existing (identical with this commit stashed), so the probe's synthetic path never reaches the kq decode rail on that model; it cannot gate this change either way. The tier arms are what covers these kernels. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

…trument Phase 7's pp half is diagnosed but not fixed; this lands the instrument that decides whether it is worth fixing. The k/v roles ride the aligned-L 128 tile, and at d = kv_dim 1024 / cnt 512 that is ceil(1024/128) * ceil(512/128) = 32 workgroups for 36 SMs — one starved wave with no latency hiding (q gets 80, gate/up 304). The two real fixes are both invasive: split-K (atomics or a two-pass reduce plus scratch), or merging k+v into ONE two-region dispatch, which today is blocked by "the batch GEMM can't offset its output independently of its input row" — the reason pf_v exists and is copied into pf_kv's half — so it needs a per-region y column offset threaded through every format's batch kernel. Ceiling is roughly 4.7ms out of a 117ms pp512, so ~+4% pp. Rather than pay that on a hypothesis, mm_small_d() reads DASLLAMA_MM_SMALLD (default 64 = the current cutoff, so routing is bit-for-bit unchanged when unset) and the tile router compares against it instead of the literal. Setting it above kv_dim sends exactly the k/v roles to the 32-tile's much wider grid, which confirms or kills the starvation hypothesis with one env var. Same shape as the existing DASLLAMA_MM_SMALL experiment arm, and still pure in (d, cnt, mode) so the pipe pick and the meta fill cannot disagree. NOT YET MEASURED: the A/B was attempted and the numbers are garbage — the GPU had already dropped off the bus (nvidia-smi "GPU is lost", vulkaninfo "No devices were found"), so both arms silently ran CPU-only. Re-run it as the first thing after the box reboots. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

@6000

DASLLAMA_MM_SMALLD's comment described an open experiment; it is closed, and negative. Recording the result where the next reader will hit it, so nobody re-runs it or reaches for split-K. Measured Qwen3-4B Q8 @6000, lcpp_bench 5-rep, engagement confirmed per arm via the "mul_mm M-tile engaged" log: L128 (shipped) pp512 4297.02 +- 36.83 M64 SMALLD=1024 MM_SMALL=64 pp512 ~4101 -4.6% sdot4-32 SMALLD=1024 pp512 3906 -9.0% The first pass only ran the sdot4 arm and was CONFOUNDED — mm_small_tile() defaults to 32 and the 32 tier IS the sdot4 kernel, so it varied tile width AND tensor cores together. The M64 arm holds coopmat constant (same 16x16 fragments, 4x the k/v grid: 32 -> 128 wgs) and still loses. So k/v is bound by the L-tile's arithmetic efficiency, not by grid starvation, despite running only 32 workgroups on 36 SMs. That also retires split-K for these shapes by implication — its premise was the same "widen the grid while keeping coopmat" — so the per-region y column offset refactor across every format's batch kernel is off the table. tg is a wash throughout, as expected for a prefill-only knob. Knob kept as an instrument (default 64 = shipped routing, inert). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

…2-arc # Conflicts: # modules/dasLLAMA/.das_module # modules/dasLLAMA/dasllama/dasllama_common.das # modules/dasLLAMA/dasllama/dasllama_math_vulkan.das

The merge with master pulled in PR #3580's contracts, and the arc's code predates every one of them. This is the follow-through, not new behaviour. Lint (28 changed files, 66 warnings -> 0): * STYLE036 x40 — `-const` is a no-op on an already-resolved cast type; dropped from reinterpret<void? -const> / addr<void? -const>. Compile-verified as a genuine no-op. * LINT017 x14 — int64(length(x)) still hits the 2^31 limit inside length() before the cast, so the widening is a lie; now long_length(x). * LINT018 x3 — memcpy grew uint/int64/uint64 overloads, so the int() size truncation is gone. * STYLE014/015 x6, STYLE027 x2, PERF006 — comment blocks trimmed to the caps, and the bench's two push-loops became comprehensions. Env registry (test_env_registry 14/14): every DASLLAMA_* read must have an entry and go through the typed readers. Registered DASLLAMA_MM_SMALLD / VK_FUSE / VK_XFERQ / VK_MEMPRIO (the arc's own knobs) plus CONV_PROF, ALLOW_INTERP_LOAD, PROBE_PATH and PROBE_GB; converted the vulkan reads to env_flag/env_int64/env_str, deleted write_cliff_probe's ad-hoc env_or, and regenerated ENVIRONMENT.md. Worth noting for the next merge: dasllama_layout.das carried 6 of these warnings because the arc extracted that code out of dasllama_common.das BEFORE master's sweep ran, so it missed the pass its siblings got. A diff-opcode scan confirmed master's 82 hunks in common.das touch none of the extracted lines, so the extraction itself lost nothing. Gates: lint 28 files / 0 issues; test_env_registry 14/14; test_vulkan_tier_cm2 6/6; test_vulkan_tier 33/34 (the 1 is fix-list #5's npos=1100 transient, unrelated); format clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

Copilot

Pull request overview

This PR advances the dasLLAMA Vulkan-tier “decode arc” and supporting tooling: it adds new GPU-facing layout/gather infrastructure, expands Vulkan/CM2 test coverage, and hardens both the MCP supervisor and standalone llvm -exe runtime behavior so failures surface deterministically.

Changes:

Hardened MCP stdio supervision: newest-binary selection per spawn and stdout “noise” filtering to prevent non-JSON diagnostics from corrupting JSON-RPC framing.
Added a guarded standalone-exe entry path (jit_run_main_guarded) and routed llvm_exe.das-generated main through it so panics print and exit non-zero.
Introduced dasllama_layout (disk→compute transforms, repack walkers, GPU tier gathers incl. q8n) plus expanded Vulkan/CM2 tests and benchmarking/profiling capture improvements.

Reviewed changes

Copilot reviewed 33 out of 34 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
utils/mcp/mcp_supervisor.py	Re-picks the newest daslang binary per respawn; skips non-JSON stdout lines so compile failures don’t masquerade as JSON-RPC frames.
utils/mcp/daslang-mcp-msvc.cmd	Defers binary choice to the supervisor (via `DASLANG_MCP_BIN`) and improves stale-binary avoidance on Windows.
src/builtin/module_jit.cpp	Adds `jit_run_main_guarded` catch boundary for standalone exe entrypoints (panic prints + nonzero exit).
modules/dasLLVM/daslib/llvm_exe.das	Generates standalone `main` to call `jit_run_main_guarded` and return a consistent exit code.
modules/dasLLAMA/tests/test_vulkan_tier.das	Pins the reference GEMM mode for deterministic CPU-reference tolerances.
modules/dasLLAMA/tests/test_vulkan_tier_cm2.das	New CM2 (NV_cooperative_matrix2) tier test suite with CPU references + streamed/arena arms.
modules/dasLLAMA/tests/test_mtp.das	Adds a non-recurrent GLM NextN/MTP invariance test to cover the reject-path branch.
modules/dasLLAMA/tests/test_metal_support_matrix.das	Pulls in `dasllama_layout` for metal blob-flavor transform coverage.
modules/dasLLAMA/tests/test_metal_prefill_parity.das	Pulls in `dasllama_layout` for metal blob-flavor transform coverage.
modules/dasLLAMA/tests/test_metal_decode_parity.das	Pulls in `dasllama_layout` for metal blob-flavor transform coverage.
modules/dasLLAMA/performance/profile_common.das	Captures env-var “arming” state and adds helpers to record real argv + env prefix for reproducible bench rows.
modules/dasLLAMA/harness/parity.das	Requires `dasllama_layout` so harness paths can use the new transform entrypoints.
modules/dasLLAMA/harness/gen_tune_probe.das	Fixes q51 tuning MR selection via layout registry + improves gate-failure fallback behavior.
modules/dasLLAMA/ENVIRONMENT.md	Documents new env knobs (conversion profiling, interp-load guard, Vulkan fuse/xferq/memprio, probes).
modules/dasLLAMA/dasllama/dasllama_transformer.das	Side-effect require of `dasllama_layout` to ensure layout hooks register at init.
modules/dasLLAMA/dasllama/dasllama_tokenizer.das	Uses `var inscope` for GGUF meta/token arrays to ensure scope-finalization.
modules/dasLLAMA/dasllama/dasllama_qwen3a.das	Uses `var inscope` for GGUF meta parsing lifetime management.
modules/dasLLAMA/dasllama/dasllama_math.das	Adds q8n probe/tag/stream-need hooks and allows uploads with empty scale plane (nullable ws pointer).
modules/dasLLAMA/dasllama/dasllama_layout.das	New module: metal blob transform, CPU repack walkers, and GPU tier gather implementations (incl. q8n).
modules/dasLLAMA/dasllama/dasllama_kernel_access.das	Extends access analysis for new coopmat tensor load/store intrinsics.
modules/dasLLAMA/dasllama/dasllama_image.das	Updates image versioning + adds interp-load guard + Vulkan-flavor image bake/mapping flow.
modules/dasLLAMA/dasllama/dasllama_gemma4a.das	Uses `var inscope` for GGUF meta parsing lifetime management.
modules/dasLLAMA/dasllama/dasllama_gemm_schema.das	Centralizes key plane-stride constants (q8, kq, q51, q8n) for shared sizing logic.
modules/dasLLAMA/dasllama/dasllama_env.das	Registers new env knobs (Vulkan tier, profiling, interp-load guard, probe params).
modules/dasLLAMA/dasllama/dasllama_bpe.das	Uses `var inscope` for GGUF meta/token arrays to ensure scope-finalization.
modules/dasLLAMA/dasllama/dasllama_audio.das	Uses `var inscope` for GGUF meta parsing lifetime management.
modules/dasLLAMA/dasllama/dasllama_asr.das	Uses `var inscope` for GGUF meta parsing lifetime management.
modules/dasLLAMA/benchmarks/write_cliff_probe.das	New standalone sustained-write probe benchmark driven by `PROBE_*` env knobs.
modules/dasLLAMA/benchmarks/lcpp_bench.das	Adds real-token tg row + MTP A/B lever + multi-model batching + bench env/cmd capture.
modules/dasLLAMA/benchmarks/decode_step_trace.das	Requires `dasllama_layout` for metal blob-flavor transform availability.
modules/dasLLAMA/.das_module	Exposes `dasllama_layout` as part of the module’s install paths.
CLAUDE.md	Updates guidance on `var inscope` finalization behavior for owned locals (arrays/structs included).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…whitespace Three accepted review findings, all real: * DASLLAMA_ALLOW_INTERP_LOAD was registered as EnvKind.flag but read with == "1", so the registry's own spellings ("on", "true", "yes") silently meant disabled. That contradiction is one I introduced by registering the knob without converting its read; now env_flag. * lcpp_bench's `toks` is an array<array<int64>> holding a whole tokenized corpus, declared inside the per-model loop the -m list added. A plain `var` does not finalize at scope exit, so every extra model leaked one corpus; now `var inscope`. * `-m "a.gguf, b.gguf"` is the natural spelling and the split left a leading space, which fails to open. Each entry is stripped before the empty check. Gates: lint 0 issues, test_env_registry 14/14, test_vulkan_tier_cm2 6/6. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

Copilot

Pull request overview

Copilot reviewed 33 out of 34 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (16)

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:224

These arrays are allocated in the test but not deleted on exit paths; unlike var inscope, plain var arrays do not finalize at scope exit. Mark them inscope to avoid leaks on success and early return paths.

            var wq1 : array<int8>
            var ws1 : array<float>
            var wq3 : array<int8>
            var ws3 : array<float>
            var wq2 : array<int8>
            var ws2 : array<float>

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:229

wqb/no_ws are heap arrays and should be freed even when the test returns early. Mark them inscope to avoid leaks.

            var wqb : array<uint8>
            var no_ws : array<uint8>   // q8n stacks upload with an EMPTY scale plane

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:246

xq/xs are allocated in the test and not deleted; mark them inscope so they are finalized when the test scope exits (including on feint/return).

            var xq : array<int8>
            var xs : array<float>

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:250

The region-offset arrays created by make_offs(...) are never deleted. Mark them inscope so the test doesn't leak per-run allocations.

            var offs1 <- make_offs([1l, 2l], [0l, 1l], [1l, 1l], WOFF1, ege1)
            var offs3 <- make_offs([1l, 2l], [0l, 1l], [1l, 1l], WOFF3, ege1)
            var offs2 <- make_offs([1l, 2l], [0l, 1l], [1l, 1l], WOFF2, ege2)

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:251

gout is a heap array and is never deleted in this test function. Mark it inscope so it is finalized on scope exit.

            var gout : array<float>

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:254

refv is allocated in the test but never deleted. Mark it inscope to avoid leaking memory when the test returns early or completes.

            var refv : array<float>

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:267

The batch-arm offset arrays (boffs*) are allocated but never deleted. Mark them inscope so they are freed when the test scope exits.

            var boffs1 <- make_offs([0l, 2l], [0l, 24l], [24l, 16l], WOFF1, ege1)
            var boffs3 <- make_offs([0l, 2l], [0l, 24l], [24l, 16l], WOFF3, ege1)
            var boffs2 <- make_offs([0l, 2l], [0l, 24l], [24l, 16l], WOFF2, ege2)

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:281

The multi-tile offset arrays (moffs*) are allocated but never deleted. Mark them inscope to avoid leaking memory across the test process.

            var moffs1 <- make_offs([0l, 1l], [0l, 130l], [130l, 70l], WOFF1, ege1)
            var moffs3 <- make_offs([0l, 1l], [0l, 130l], [130l, 70l], WOFF3, ege1)
            var moffs2 <- make_offs([0l, 1l], [0l, 130l], [130l, 70l], WOFF2, ege2)

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:375

wqb/no_ws are heap arrays and should be freed even on the early-return paths in this test. Mark them inscope.

            var wqb : array<uint8>
            var no_ws : array<uint8>

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:389

xq/xs are allocated in the streamed test but not deleted; mark them inscope to avoid leaks.

            var xq : array<int8>
            var xs : array<float>

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:393

The streamed test allocates offset arrays via make_offs(...) but never deletes them. Mark them inscope to free them at scope exit.

            var offs1 <- make_offs([0l, 2l], [0l, 24l], [24l, 16l], SWOFF, ege1)
            var offs3 <- make_offs([0l, 2l], [0l, 24l], [24l, 16l], SWOFF + 1000000l, ege1)
            var offs2 <- make_offs([0l, 2l], [0l, 24l], [24l, 16l], SWOFF + 2000000l, ege2)

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:394

gout is allocated in the streamed test but never deleted. Mark it inscope to avoid leaking memory.

            var gout : array<float>

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:397

refv is allocated in the streamed test but never deleted. Mark it inscope so it is finalized on scope exit.

            var refv : array<float>

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:426

wqb/no_ws are allocated in the arena test but never deleted. Mark them inscope to ensure they are finalized on scope exit.

            var wqb : array<uint8>
            var no_ws : array<uint8>

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:435

xq/xs are allocated in the arena test but never deleted. Mark them inscope to avoid leaking memory.

            var xq : array<int8>
            var xs : array<float>

modules/dasLLAMA/tests/test_vulkan_tier_cm2.das:437

y is a heap array and should be finalized on all exit paths of the arena test. Mark it inscope.

            var y : array<float>

Copilot round 2: 55 plain `var ... : array<T>` locals in test_vulkan_tier_cm2 never finalized, so the suite leaked its scratch across every arm. A plain `var` does not finalize at scope exit in daslang, which is exactly the per-call-array rule the tree already carries. make_offs is deliberately untouched — it does `return <- offs`, and inscope on a moved-out local double-finalizes. Verified: cm2 6/6 with no leak lines, lint clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

Copilot

Pull request overview

Copilot reviewed 33 out of 34 changed files in this pull request and generated 2 comments.

Copilot found this on the dasVulkan side and the same shape exists here — my own regression, from mechanically dropping the int() casts LINT018 flags. vk_bake_take resizes its destination planes with int(e.wq_bytes) but, after the sweep, memcpys the full int64 width. Below 2GB they agree; above it the resize truncates and the copy runs past the allocation. The comment two lines down already says multi-GB bakes are real on this path, so this was not hypothetical. Destinations are int-indexed arrays and cannot hold a >2GB plane at all, so assert rather than silently truncate. Audited every other site the sweep touched. vk_bake_append is the opposite case and correct as it stands: need/abytes are int64 and the blob deliberately uses the long_ forms, so dropping int() there FIXED a latent truncation on multi-GB appends. The two test_compute_shared copies kept their counts unchanged. Gates: compiles, lint 0 issues, cm2 6/6, env registry 14/14. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

Two accepted findings. **DASLANG_MCP_BIN was a hard pin with no existence check**, so a typo'd path was handed straight to Popen as a bare FileNotFoundError — or written into .mcp.json — instead of naming the bad path. That directly contradicts the commit that introduced it, whose whole point was that this supervisor's failures should be actionable. A missing pin is now announced (log + stderr) and ignored in favour of auto-pick, so the server still comes up and the reason is visible. A pin that exists is honoured exactly as before; both paths probed. **shell_quote only triggered on spaces and never escaped**, while its callers' docstrings promise a copy-pasteable reproduction line. Full cross-shell escaping is not well-defined — cmd and POSIX disagree on the rules — so this does not attempt it; it widens the trigger to anything outside a conservative safe set, which is what stops a bare `;`, `&` or `$` in a value from pasting as shell syntax rather than data. Realistic inputs here are model paths and numeric knobs, so this is narrowing a hazard rather than closing it, and the comment says so. Gates: both compile, lint 0 issues. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

Copilot

Pull request overview

Copilot reviewed 33 out of 34 changed files in this pull request and generated 3 comments.

Copilot round 4, three threads on one issue. The write-cliff probe defaulted to `D:/_wcliff.bin`, which on POSIX is not an absolute path at all — it is a relative `D:` directory that does not exist, so the probe fails confusingly on every non-Windows host rather than writing anywhere. Default is now cwd-relative `_wcliff.bin`, which resolves on every host; the registry entry and the usage line say to point it at the drive under test, which is the whole point of the knob. Also dropped `require strings`, unused since env_int64 replaced the to_int call. Gates: compiles, lint 0 issues, test_env_registry 14/14, ENVIRONMENT.md regenerated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

Copilot

Pull request overview

Copilot reviewed 33 out of 34 changed files in this pull request and generated 2 comments.

Two accepted findings, both mine. _with_noise sliced only the joined noise, so the returned string could still exceed the cap by whatever the reason carried — and that reason can be an arbitrarily long OSError, inside a JSON-RPC error the client has to render. The cap now applies to the whole message on both branches. The four knobs I added were registered inside reg_vulkan even though two are EnvArea.engine and two are EnvArea.benchmark. env_registry is the single source rendered in documentation order, so area-mismatched entries drift ENVIRONMENT.md's grouping and make the file harder to navigate. Moved CONV_PROF / ALLOW_INTERP_LOAD to reg_engine and PROBE_PATH / PROBE_GB to reg_benchmark. I put them where I happened to be editing rather than where they belonged. Gates: test_env_registry 14/14, ENVIRONMENT.md regenerates byte-identical, lint 0 issues, the cap probed at 2000 with and without noise. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

Copilot

Pull request overview

Copilot reviewed 33 out of 34 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 33 out of 34 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

utils/mcp/mcp_supervisor.py:127

On POSIX, _spawn_and_replay re-resolves picked each spawn but still runs self.launcher (which was built once in _default_launcher). That means a rebuild that makes a different output layout newest (e.g. build/daslang -> bin/Release/daslang) will keep respawning the old binary, reintroducing the stale-build-id failure the comment is trying to avoid. The DASLANG_MCP_BIN env var only affects the Windows .cmd launcher, not a direct POSIX binary exec.

The per-spawn re-resolve only took effect on Windows, where the vcvars .cmd reads DASLANG_MCP_BIN. On POSIX the launcher argv is built once at construction and execs the binary directly, so nothing read the variable: a rebuild that changed which layout is newest would not apply mid-session, and the spawn log named a binary that had not been executed. argv[0] is now replaced with the pick on non-Windows, and the log reports what actually ran. A misleading diagnostic is the specific failure this supervisor's changes exist to prevent, so this one mattered more than its size. Verified: the supervisor still serves a full tools/list (43 tools). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ruKZPG8G7SPEUyZtvBzcT

Copilot

Pull request overview

Copilot reviewed 33 out of 34 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

modules/dasLLAMA/benchmarks/write_cliff_probe.das:22

Defaulting PROBE_PATH to a cwd-relative filename can leave a very large scratch file in the working tree if the probe is interrupted or fails before cleanup. Since fio.temp_directory is portable and already falls back when needed, consider using it for the default and keep PROBE_PATH as the opt-in override for “drive under test”.

    let gb = max(env_int64("PROBE_GB", 50l), 1l)
    // cwd-relative so the default works on every host; point it at the drive under test
    let path = env_str("PROBE_PATH", "_wcliff.bin")

borisbat and others added 28 commits July 26, 2026 09:03

Merge remote-tracking branch 'origin/master' into bbatkin/dasllama-cm…

bc9249d

…2-arc # Conflicts: # modules/dasLLAMA/.das_module # modules/dasLLAMA/dasllama/dasllama_common.das # modules/dasLLAMA/dasllama/dasllama_math_vulkan.das

Copilot AI review requested due to automatic review settings July 27, 2026 18:01

Copilot started reviewing on behalf of borisbat July 27, 2026 18:02 View session